Skip to main content
SpringerLink
Log in

Test case prioritization using test case diversification and fault-proneness estimations

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Regression testing activities greatly reduce the risk of faulty software release. However, the size of the test suites grows throughout the development process, resulting in time-consuming execution of the test suite and delayed feedback to the software development team. This has urged the need for approaches such as test case prioritization (TCP) and test-suite reduction to reach better results in case of limited resources. In this regard, proposing approaches that use auxiliary sources of data such as bug history can be interesting. We aim to propose an approach for TCP that takes into account test case coverage data, bug history, and test case diversification. To evaluate this approach we study its performance on real-world open-source projects. The bug history is used to estimate the fault-proneness of source code areas. The diversification of test cases is preserved by incorporating fault-proneness on a clustering-based approach scheme. The proposed methods are evaluated on datasets collected from the development history of five real-world projects including 357 versions in total. The experiments show that the proposed methods are superior to coverage-based TCP methods. The proposed approach shows that improvement of coverage-based and fault-proneness-based methods is possible by using a combination of diversification and fault-proneness incorporation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. This relation is expressed using Z notation’s first order logic (Woodcock and Davies 1996).

  2. \(F_{j}\) indicates the event in which jth code unit is faulty and \(Prob(F_J)\) represents the probability of this event.

  3. https://github.com/khesoem/Defects4J-Plus-M.

  4. https://github.com/mostafamahdieh/ClusteringFaultPronenessTCP.

References

  • Abou Assi, R., Masri, W., Trad, C.: How detrimental is coincidental correctness to coverage-based fault detection and localization? An empirical study. Softw. Test. Verif. Reliab. 31(5), 1762 (2021)

    Article  Google Scholar 

  • Alves, E.L., Machado, P.D., Massoni, T., Kim, M.: Prioritizing test cases for early detection of refactoring faults. Softw. Test. Verif. Reliab. 26(5), 402–426 (2016)

    Article  Google Scholar 

  • Bagherzadeh, M., Kahani, N., Briand, L.: Reinforcement learning for test case prioritization. IEEE Trans. Softw. Eng. (2021). https://doi.org/10.48550/arXiv.2011.01834

    Article  Google Scholar 

  • Bansiya, J., Davis, C.G.: A hierarchical model for object-oriented design quality assessment. IEEE Trans. Softw. Eng. 28(1), 4–17 (2002)

    Article  Google Scholar 

  • Bishnu, P.S., Bhattacherjee, V.: Software fault prediction using quad tree-based k-means clustering algorithm. IEEE Trans. Knowl. Data Eng. 24(6), 1146–1150 (2011)

    Article  Google Scholar 

  • Boucher, A., Badri, M.: Software metrics thresholds calculation techniques to predict fault-proneness: an empirical comparison. Inf. Softw. Technol. 96, 38–67 (2018)

    Article  Google Scholar 

  • Boughorbel, S., Jarray, F., El-Anbari, M.: Optimal classifier for imbalanced data using Matthews correlation coefficient metric. PLoS ONE 12(6), 0177678 (2017)

    Article  Google Scholar 

  • Catal, C., Mishra, D.: Test case prioritization: a systematic mapping study. Softw. Qual. J. 21(3), 445–478 (2013)

    Article  Google Scholar 

  • Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  MATH  Google Scholar 

  • Chen, J., Kuo, F.-C., Chen, T.Y., Towey, D., Su, C., Huang, R.: A similarity metric for the inputs of OO programs and its application in adaptive random testing. IEEE Trans. Reliab. 66(2), 373–402 (2016)

    Article  Google Scholar 

  • Chen, J., Zhu, L., Chen, T.Y., Towey, D., Kuo, F.-C., Huang, R., Guo, Y.: Test case prioritization for object-oriented software: an adaptive random sequence approach based on clustering. J. Syst. Softw. 135, 107–125 (2018)

    Article  Google Scholar 

  • Chicco, D.: Ten quick tips for machine learning in computational biology. BioData Min. 10(1), 1–17 (2017)

    Article  Google Scholar 

  • Chidamber, S.R., Kemerer, C.F.: A metrics suite for object oriented design. IEEE Trans. Softw. Eng. 20(6), 476–493 (1994)

    Article  Google Scholar 

  • D’Ambros, M., Lanza, M., Robbes, R.: Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empir. Softw. Eng. 17(4), 531–577 (2012)

    Article  Google Scholar 

  • Davies, D.L., Bouldin, D.W.: A cluster separation measure. IEEE Trans. Pattern Anal. Mach Intell. 2, 224–227 (1979)

    Article  Google Scholar 

  • Deng, J., Lu, L., Qiu, S.: Software defect prediction via LSTM. IET Softw. 14(4), 443–450 (2020)

    Article  Google Scholar 

  • e Abreu, F.B., Carapuça, R.: Candidate metrics for object-oriented software within a taxonomy framework. J. Syst. Softw. 26(1), 87–96 (1994)

    Article  Google Scholar 

  • Elbaum, S., Malishevsky, A.G., Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)

    Article  Google Scholar 

  • Elbaum, S., Rothermel, G., Kanduri, S., Malishevsky, A.G.: Selecting a cost-effective test case prioritization technique. Softw. Qual. J. 12(3), 185–210 (2004)

    Article  Google Scholar 

  • Elish, K.O., Elish, M.O.: Predicting defect-prone software modules using support vector machines. J. Syst. Softw. 81(5), 649–660 (2008)

    Article  Google Scholar 

  • Fang, C., Chen, Z., Wu, K., Zhao, Z.: Similarity-based test case prioritization using ordered sequences of program entities. Softw. Qual. J. 22(2), 335–361 (2014)

    Article  Google Scholar 

  • Fu, W., Yu, H., Fan, G., Ji, X.: Coverage-based clustering and scheduling approach for test case prioritization. IEICE Trans. Inf. Syst. 100(6), 1218–1230 (2017)

    Article  Google Scholar 

  • Graves, T.L., Karr, A.F., Marron, J.S., Siy, H.: Predicting fault incidence using software change history. IEEE Trans. Softw. Eng. 26(7), 653–661 (2000)

    Article  Google Scholar 

  • Grindal, M., Lindström, B., Offutt, J., Andler, S.F.: An evaluation of combination strategies for test case selection. Empir. Softw. Eng. 11(4), 583–611 (2006)

    Article  Google Scholar 

  • Halstead, M.H.: Elements of Software Science. Operating and Programming Systems Series, Elsevier Science, Inc., Amsterdam (1977)

    MATH  Google Scholar 

  • Hao, D., Zhang, L., Zhang, L., Rothermel, G., Mei, H.: A unified test case prioritization approach. ACM Trans. Softw. Eng. Methodol. 24(2), 1–31 (2014)

    Article  Google Scholar 

  • Hao, D., Zhang, L., Zang, L., Wang, Y., Wu, X., Xie, T.: To be optimal or not in test-case prioritization. IEEE Trans. Softw. Eng. 42(5), 490–505 (2015)

    Article  Google Scholar 

  • Harrison, R., Counsell, S.J., Nithi, R.V.: An evaluation of the mood set of object-oriented software metrics. IEEE Trans. Softw. Eng. 24(6), 491–496 (1998)

    Article  Google Scholar 

  • Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, 2nd edn. Springer, Berlin (2008). (Google Scholar)

    MATH  Google Scholar 

  • Hettiarachchi, C., Do, H., Choi, B.: Risk-based test case prioritization using a fuzzy expert system. Inf. Softw. Technol. 69, 1–15 (2016)

    Article  Google Scholar 

  • Jones, J.A., Harrold, M.J.: Test-suite reduction and prioritization for modified condition/decision coverage. IEEE Trans. Softw. Eng. 29(3), 195–209 (2003)

    Article  Google Scholar 

  • Kamei, Y., Shihab, E., Adams, B., Hassan, A.E., Mockus, A., Sinha, A., Ubayashi, N.: A large-scale empirical study of just-in-time quality assurance. IEEE Trans. Softw. Eng. 39(6), 757–773 (2012)

    Article  Google Scholar 

  • Kandil, P., Moussa, S., Badr, N.: Cluster-based test cases prioritization and selection technique for agile regression testing. J. Softw. Evol. Process 29(6), 1794 (2017)

    Article  Google Scholar 

  • Kanmani, S., Uthariaraj, V.R., Sankaranarayanan, V., Thambidurai, P.: Object-oriented software fault prediction using neural networks. Inf. Softw. Technol. 49(5), 483–492 (2007)

    Article  Google Scholar 

  • Kazmi, R., Jawawi, D.N., Mohamad, R., Ghani, I.: Effective regression test case selection: a systematic literature review. ACM Comput. Surv. (CSUR) 50(2), 1–32 (2017)

    Article  Google Scholar 

  • Khalilian, A., Azgomi, M.A., Fazlalizadeh, Y.: An improved method for test case prioritization by incorporating historical test case data. Sci. Comput. Program. 78(1), 93–116 (2012)

    Article  MATH  Google Scholar 

  • Khatibsyarbini, M., Isa, M.A., Jawawi, D.N., Tumeng, R.: Test case prioritization approaches in regression testing: a systematic literature review. Inf. Softw. Technol. 93, 74–93 (2018)

    Article  Google Scholar 

  • Kumar, A.: Development at the Speed and Scale of Google. QCon, San Francisco (2010)

    Google Scholar 

  • Ledru, Y., Petrenko, A., Boroday, S., Mandran, N.: Prioritizing test cases with string distances. Autom. Softw. Eng. 19(1), 65–95 (2012)

    Article  Google Scholar 

  • Lessmann, S., Baesens, B., Mues, C., Pietsch, S.: Benchmarking classification models for software defect prediction: a proposed framework and novel findings. IEEE Trans. Softw. Eng. 34(4), 485–496 (2008)

    Article  Google Scholar 

  • Li, Z., Jing, X.-Y., Zhu, X.: Progress on approaches to software defect prediction. IET Softw. 12(3), 161–175 (2018)

    Article  Google Scholar 

  • Li, N., Shepperd, M., Guo, Y.: A systematic review of unsupervised learning techniques for software defect prediction. Inf. Softw. Technol. 122, 106287 (2020)

    Article  Google Scholar 

  • Liang, H., Yu, Y., Jiang, L., Xie, Z.: Seml: a semantic LSTM model for software defect prediction. IEEE Access 7, 83812–83824 (2019)

    Article  Google Scholar 

  • Luo, Q., Moran, K., Zhang, L., Poshyvanyk, D.: How do static and dynamic test case prioritization techniques perform on modern software systems? An extensive study on GitHub projects. IEEE Trans. Softw. Eng. 45(11), 1054–1080 (2018a)

    Article  Google Scholar 

  • Mahdieh, M., Mirian-Hosseinabadi, S.-H., Etemadi, K., Nosrati, A., Jalali, S.: Incorporating fault-proneness estimations into coverage-based test case prioritization methods. Inf. Softw. Technol. 121, 106269 (2020)

    Article  Google Scholar 

  • Majd, A., Vahidi-Asl, M., Khalilian, A., Poorsarvi-Tehrani, P., Haghighi, H.: SLDeep: statement-level software defect prediction using deep-learning model on static code features. Expert Syst. Appl. 147, 113156 (2020)

    Article  Google Scholar 

  • Mathur, A.P.: Foundations of Software Testing. Addison-Wesley Professional 11. Academic Accommodation Policy. Addison-Wesley, Boston (2010)

    Google Scholar 

  • Matloob, F., Ghazal, T.M., Taleb, N., Aftab, S., Ahmad, M., Khan, M.A., Abbas, S., Soomro, T.R.: Software defect prediction using ensemble learning: a systematic literature review. IEEE Access 9, 98754–98771 (2021)

    Article  Google Scholar 

  • McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. 4, 308–320 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  • Mei, H., Hao, D., Zhang, L., Zhang, L., Zhou, J., Rothermel, G.: A static approach to prioritizing JUnit test cases. IEEE Trans. Softw. Eng. 38(6), 1258–1275 (2012)

    Article  Google Scholar 

  • Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. Softw. Eng. 33(1), 2–13 (2006)

    Article  Google Scholar 

  • Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. Softw. Eng. 33(1), 2–13 (2007)

    Article  Google Scholar 

  • Menzies, T., Milton, Z., Turhan, B., Cukic, B., Jiang, Y., Bener, A.: Defect prediction from static code features: current results, limitations, new approaches. Autom. Softw. Eng. 17(4), 375–407 (2010)

    Article  Google Scholar 

  • Okutan, A., Yıldız, O.T.: Software defect prediction using Bayesian networks. Empir. Softw. Eng. 19(1), 154–181 (2014)

    Article  Google Scholar 

  • Ostrand, T.J., Weyuker, E.J., Bell, R.M.: Predicting the location and number of faults in large software systems. IEEE Trans. Softw. Eng. 31(4), 340–355 (2005)

    Article  Google Scholar 

  • Pan, R., Bagherzadeh, M., Ghaleb, T.A., Briand, L.: Test case selection and prioritization using machine learning: a systematic literature review. Empir. Softw. Eng. 27(2), 1–43 (2022)

    Article  Google Scholar 

  • Panda, S., Munjal, D., Mohapatra, D.P.: A slice-based change impact analysis for regression test case prioritization of object-oriented programs. Adv. Softw. Eng. (2016). https://doi.org/10.1155/2016/7132404

    Article  Google Scholar 

  • Pandey, S.K., Mishra, R.B., Tripathi, A.K.: BPDET: an effective software bug prediction model using deep representation and ensemble learning techniques. Expert Syst. Appl. 144, 113085 (2020)

    Article  Google Scholar 

  • Pei, H., Yin, B., Xie, M., Cai, K.-Y.: Dynamic random testing with test case clustering and distance-based parameter adjustment. Inf. Softw. Technol. 131, 106470 (2021)

    Article  Google Scholar 

  • Pradhan, D., Wang, S., Ali, S., Yue, T., Liaaen, M.: Employing rule mining and multi-objective search for dynamic test case prioritization. J. Syst. Softw. 153, 86–104 (2019)

    Article  Google Scholar 

  • Rahman, M.A., Hasan, M.A., Siddik, M.S.: Prioritizing dissimilar test cases in regression testing using historical failure data. Int. J. Comput. Appl. 975, 8887 (2018)

    Google Scholar 

  • Rothermel, G., Harrold, M.J., Von Ronne, J., Hong, C.: Empirical studies of test-suite reduction. Softw. Test. Verif. Reliab. 12(4), 219–249 (2002)

    Article  Google Scholar 

  • Shrivathsan, A., Ravichandran, K., Krishankumar, R., Sangeetha, V., Kar, S., Ziemba, P., Jankowski, J.: Novel fuzzy clustering methods for test case prioritization in software projects. Symmetry 11(11), 1400 (2019)

    Article  Google Scholar 

  • Song, Q., Guo, Y., Shepperd, M.: A comprehensive investigation of the role of imbalanced learning for software defect prediction. IEEE Trans. Softw. Eng. 45(12), 1253–1269 (2018)

    Article  Google Scholar 

  • Srikanth, H., Hettiarachchi, C., Do, H.: Requirements based test prioritization using risk factors: an industrial study. Inf. Softw. Technol. 69, 71–83 (2016)

    Article  Google Scholar 

  • Tan, P.-N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Pearson Education, Noida (2016)

    Google Scholar 

  • Tong, H., Liu, B., Wang, S.: Software defect prediction using stacked denoising autoencoders and two-stage ensemble learning. Inf. Softw. Technol. 96, 94–111 (2018)

    Article  Google Scholar 

  • Wang, R., Jiang, S., Chen, D., Zhang, Y.: Empirical study of the effects of different similarity measures on test case prioritization. Math. Probl. Eng. (2016a). https://doi.org/10.1155/2016/8343910

    Article  Google Scholar 

  • Wang, T., Zhang, Z., Jing, X., Liu, Y.: Non-negative sparse-based semiboost for software defect prediction. Softw. Test. Verif. Reliab. 26(7), 498–515 (2016b)

    Article  Google Scholar 

  • Wang, S., Liu, T., Nam, J., Tan, L.: Deep semantic feature learning for software defect prediction. IEEE Trans. Softw. Eng. 46(12), 1267–1293 (2018)

    Article  Google Scholar 

  • Weyuker, E.J., Ostrand, T.J., Bell, R.M.: Do too many cooks spoil the broth? Using the number of developers to enhance defect prediction models. Empir. Softw. Eng. 13(5), 539–559 (2008)

    Article  Google Scholar 

  • Woodcock, J., Davies, J.: Using Z: Specification, Refinement, and Proof, vol. 39. Prentice Hall, Englewood Cliffs (1996)

    MATH  Google Scholar 

  • Xu, D., Tian, Y.: A comprehensive survey of clustering algorithms. Ann. Data Sci. 2(2), 165–193 (2015)

    Article  MathSciNet  Google Scholar 

  • Xu, Z., Li, S., Xu, J., Liu, J., Luo, X., Zhang, Y., Zhang, T., Keung, J., Tang, Y.: LDFR: learning deep feature representation for software defect prediction. J. Syst. Softw. 158, 110402 (2019)

    Article  Google Scholar 

  • Yao, J., Shepperd, M.: The impact of using biased performance metrics on software defect prediction research. Inf. Softw. Technol. 139, 106664 (2021)

    Article  Google Scholar 

  • Yedida, R., Menzies, T.: On the value of oversampling for deep learning in software defect prediction. IEEE Trans. Softw. Eng. (2021). https://doi.org/10.48550/arXiv.2008.03835

    Article  Google Scholar 

  • Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verif. Reliab. 22(2), 67–120 (2012)

    Article  Google Scholar 

  • Zhang, Z.-W., Jing, X.-Y., Wang, T.-J.: Label propagation based semi-supervised learning for software defect prediction. Autom. Softw. Eng. 24(1), 47–69 (2017)

    Article  Google Scholar 

  • Zhong, H., Zhang, L., Mei, H.: An experimental study of four typical test suite reduction techniques. Inf. Softw. Technol. 50(6), 534–546 (2008)

    Article  Google Scholar 

  • Zhu, K., Zhang, N., Ying, S., Zhu, D.: Within-project and cross-project just-in-time defect prediction based on denoising autoencoder and convolutional neural network. IET Softw. 14(3), 185–195 (2020)

    Article  Google Scholar 

  • Aljamaan, H., Alazba, A.: Software defect prediction using tree-based ensembles. In: Proceedings of the 16th ACM International Conference on Predictive Models and Data Analytics in Software Engineering, pp. 1–10 (2020)

  • Anderson, J., Salem, S., Do, H.: Improving the effectiveness of test suite through mining historical data. In: Proceedings of the 11th Working Conference on Mining Software Repositories, pp. 142–151. ACM (2014)

  • Arafeen, M.J., Do, H.: Test case prioritization using requirements-based clustering. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, pp. 312–321. IEEE (2013)

  • Ashraf, E., Rauf, A., Mahmood, K.: Value based regression test case prioritization. In: Proceedings of the World Congress on Engineering and Computer Science, vol. 1, pp. 24–26 (2012)

  • Carlson, R., Do, H., Denton, A.: A clustering approach to improving test case prioritization: an industrial case study. In: ICSM, vol. 11, pp. 382–391 (2011)

  • Chen, J., Bai, Y., Hao, D., Zhang, L., Zhang, L., Xie, B.: How do assertions impact coverage-based test-suite reduction? In: 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 418–423. IEEE (2017)

  • Eghbali, S., Kudva, V., Rothermel, G., Tahvildari, L.: Supervised tie breaking in test case prioritization. In: 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 242–243. IEEE (2019)

  • Engström, E., Runeson, P., Wikstrand, G.: An empirical evaluation of regression testing based on fix-cache recommendations. In: 2010 Third International Conference on Software Testing, Verification and Validation, pp. 75–78. IEEE (2010)

  • Engström, E., Runeson, P., Ljung, A.: Improving regression testing transparency and efficiency with history-based prioritization—an industrial case study. In: 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation, pp. 367–376. IEEE (2011)

  • Fraser, G., Wotawa, F.: Redundancy based test-suite reduction. In: International Conference on Fundamental Approaches to Software Engineering, pp. 291–305. Springer (2007)

  • Fu, W., Menzies, T.: Revisiting unsupervised learning for defect prediction. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, pp. 72–83 (2017)

  • Ghotra, B., McIntosh, S., Hassan, A.E.: Revisiting the impact of classification techniques on the performance of defect prediction models. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1, pp. 789–800. IEEE (2015)

  • Guo, L., Ma, Y., Cukic, B., Singh, H.: Robust prediction of fault-proneness by random forests. In: 15th International Symposium on Software Reliability Engineering, pp. 417–428. IEEE (2004)

  • Hasan, M.A., Rahman, M.A., Siddik, M.S.: Test case prioritization based on dissimilarity clustering using historical data analysis. In: International Conference on Information, Communication and Computing Technology, pp. 269–281. Springer (2017)

  • Hemmati, H.: Advances in techniques for test prioritization. In: Advances in Computers, vol. 112, pp. 185–221. Elsevier, Amsterdam (2019)

  • Hoang, T., Dam, H.K., Kamei, Y., Lo, D., Ubayashi, N.: DeepJIT: an end-to-end deep learning framework for just-in-time defect prediction. In: 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), pp. 34–45. IEEE (2019)

  • Hoang, T., Kang, H.J., Lo, D., Lawall, J.: CC2Vec: distributed representations of code changes. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pp. 518–529 (2020)

  • Jiang, B., Zhang, Z., Chan, W.K., Tse, T.: Adaptive random test case prioritization. In: 2009 IEEE/ACM International Conference on Automated Software Engineering, pp. 233–244. IEEE (2009)

  • Jing, X.-Y., Ying, S., Zhang, Z.-W., Wu, S.-S., Liu, J.: Dictionary learning based software defect prediction. In: Proceedings of the 36th International Conference on Software Engineering, pp. 414–423 (2014)

  • Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, pp. 437–440. ACM (2014)

  • Khoshgoftaar, T.M., Gao, K., Seliya, N.: Attribute selection and imbalanced data: problems in software defect prediction. In: 2010 22nd IEEE International Conference on Tools with Artificial Intelligence, vol. 1, pp. 137–144. IEEE (2010)

  • Kim, S., Baik, J.: An effective fault aware test case prioritization by incorporating a fault localization technique. In: Proceedings of the 2010 ACM–IEEE International Symposium on Empirical Software Engineering and Measurement, p. 5. ACM (2010)

  • Kläs, M., Elberzhager, F., Münch, J., Hartjes, K., Von Graevemeyer, O.: Transparent combination of expert and measurement data for defect prediction: an industrial case study. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, vol. 2, pp. 119–128. ACM (2010)

  • Kwon, J.-H., Ko, I.-Y., Rothermel, G., Staats, M.: Test case prioritization based on information retrieval concepts. In: 2014 21st Asia–Pacific Software Engineering Conference, vol. 1, pp. 19–26. IEEE (2014)

  • Laali, M., Liu, H., Hamilton, M., Spichkova, M., Schmidt, H.W.: Test case prioritization using online fault detection information. In: Ada-Europe International Conference on Reliable Software Technologies, pp. 78–93. Springer (2016)

  • Lachmann, R., Schulze, S., Nieke, M., Seidl, C., Schaefer, I.: System-level test case prioritization using machine learning. In: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 361–368. IEEE (2016)

  • Leon, D., Podgurski, A.: A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases. In: 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003, pp. 442–453. IEEE (2003)

  • Li, J., He, P., Zhu, J., Lyu, M.R.: Software defect prediction via convolutional neural network. In: 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 318–328. IEEE (2017)

  • Li, R., Zhou, L., Zhang, S., Liu, H., Huang, X., Sun, Z.: Software defect prediction based on ensemble learning. In: Proceedings of the 2019 2nd International Conference on Data Science and Information Technology, pp. 1–6 (2019)

  • Lou, Y., Chen, J., Zhang, L., Hao, D.: A survey on regression test-case prioritization. In: Advances in Computers, vol. 113, pp. 1–46. Elsevier, Amsterdam (2019)

  • Luo, Q., Moran, K., Poshyvanyk, D.: A large-scale empirical comparison of static and dynamic test case prioritization techniques. In: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 559–570 (2016)

  • Luo, Q., Moran, K., Poshyvanyk, D., Di Penta, M.: Assessing test case prioritization on real faults and mutants. In: 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 240–251. IEEE (2018b)

  • Memon, A., Gao, Z., Nguyen, B., Dhanda, S., Nickell, E., Siemborski, R., Micco, J.: Taming Google-scale continuous testing. In: 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP), pp. 233–242. IEEE (2017)

  • Meneely, A., Williams, L., Snipes, W., Osborne, J.: Predicting failures with developer networks and social network analysis. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 13–23. ACM (2008)

  • Miranda, B., Cruciani, E., Verdecchia, R., Bertolino, A.: Fast approaches to scalable similarity-based test case prioritization. In: 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE), pp. 222–232. IEEE (2018)

  • Mirarab, S., Tahvildari, L.: A prioritization approach for software test cases based on Bayesian networks. In: International Conference on Fundamental Approaches to Software Engineering, pp. 276–290. Springer (2007)

  • Mondal, D., Hemmati, H., Durocher, S.: Exploring test suite diversification and code coverage in multi-objective test case selection. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–10. IEEE (2015)

  • Moser, R., Pedrycz, W., Succi, G.: A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In: ACM/IEEE 30th International Conference on Software Engineering, 2008. ICSE’08. pp. 181–190. IEEE (2008)

  • Nam, J.: Survey on Software Defect Prediction. Technical Report. Department of Computer Science and Engineering, The Hong Kong University of Science and Technology (2014)

  • Nam, J., Kim, S.: CLAMI: defect prediction on unlabeled datasets. In: 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 452–463. IEEE (2015)

  • Noor, T.B., Hemmati, H.: A similarity-based approach for test case prioritization using historical failure data. In: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), pp. 58–68. IEEE (2015)

  • Noor, T.B., Hemmati, H.: Studying test case failure prediction for test case prioritization. In: Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering, pp. 2–11 (2017)

  • Palma, F., Abdou, T., Bener, A., Maidens, J., Liu, S.: An improvement to test case failure prediction in the context of test case prioritization. In: Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering, pp. 80–89 (2018)

  • Pang, Y., Xue, X., Namin, A.S.: Identifying effective test cases through k-means clustering for enhancing regression testing. In: 2013 12th International Conference on Machine Learning and Applications, vol. 2, pp. 78–83. IEEE (2013)

  • Paterson, D., Campos, J., Abreu, R., Kapfhammer, G.M., Fraser, G., McMinn, P.: An empirical study on the use of defect prediction for test case prioritization. In: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pp. 346–357. IEEE (2019)

  • Petrić, J., Bowes, D., Hall, T., Christianson, B., Baddoo, N.: Building an ensemble for software defect prediction based on diversity selection. In: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 1–10 (2016)

  • Pinzger, M., Nagappan, N., Murphy, B.: Can developer-module networks predict failures? In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 2–12. ACM (2008)

  • Saha, R.K., Zhang, L., Khurshid, S., Perry, D.E.: An information retrieval approach for regression test prioritization based on program changes. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1, pp. 268–279. IEEE (2015)

  • Salehie, M., Li, S., Tahvildari, L., Dara, R., Li, S., Moore, M.: Prioritizing requirements-based regression test cases: a goal-driven practice. In: 2011 15th European Conference on Software Maintenance and Reengineering, pp. 329–332. IEEE (2011)

  • Sandha, S.S., Aggarwal, M., Fedorov, I., Srivastava, M.: Mango: a python library for parallel hyperparameter tuning. In: ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3987–3991. IEEE (2020)

  • Shivaji, S., Whitehead, E.J., Akella, R., Kim, S.: Reducing features to improve bug prediction. In: 2009 IEEE/ACM International Conference on Automated Software Engineering, pp. 600–604. IEEE (2009)

  • Spieker, H., Gotlieb, A., Marijan, D., Mossige, M.: Reinforcement learning for automatic test case prioritization and selection in continuous integration. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 12–22 (2017)

  • Wang, S., Ali, S., Yue, T., Bakkeli, Ø., Liaaen, M.: Enhancing test case prioritization in an industrial setting with resource awareness and multi-objective search. In: Proceedings of the 38th International Conference on Software Engineering Companion, pp. 182–191 (2016c)

  • Wang, S., Liu, T., Tan, L.: Automatically learning semantic features for defect prediction. In: 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE), pp. 297–308. IEEE (2016d)

  • Wang, S., Nam, J., Tan, L.: QTEP: quality-aware test case prioritization. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, pp. 523–534 (2017)

  • Wilcoxon, F.: Individual comparisons by ranking methods. In: Breakthroughs in Statistics, pp. 196–202. Springer, New York (1992)

  • Yan, M., Fang, Y., Lo, D., Xia, X., Zhang, X.: File-level defect prediction: unsupervised vs. supervised models. In: 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 344–353. IEEE (2017)

  • Yang, X., Lo, D., Xia, X., Zhang, Y., Sun, J.: Deep learning for just-in-time defect prediction. In: 2015 IEEE International Conference on Software Quality, Reliability and Security, pp. 17–26. IEEE (2015)

  • Yang, Y., Zhou, Y., Liu, J., Zhao, Y., Lu, H., Xu, L., Xu, B., Leung, H.: Effort-aware just-in-time defect prediction: simple unsupervised models could be better than supervised models. In: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 157–168 (2016)

  • Yao, J., Shepperd, M.: Assessing software defection prediction performance: why using the Matthews correlation coefficient matters. In: Proceedings of the Evaluation and Assessment in Software Engineering, pp. 120–129 (2020)

  • Yoo, S., Harman, M.: Pareto efficient multi-objective test case selection. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, pp. 140–150 (2007)

  • Yoo, S., Harman, M., Tonella, P., Susi, A.: Clustering test cases to achieve effective and scalable prioritisation incorporating expert knowledge. In: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, pp. 201–212 (2009)

  • Zainab, A., Ghrayeb, A., Houchati, M., Refaat, S.S., Abu-Rub, H.: Performance evaluation of tree-based models for big data load forecasting using randomized hyperparameter tuning. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 5332–5339. IEEE (2020)

  • Zhang, F., Zheng, Q., Zou, Y., Hassan, A.E.: Cross-project defect prediction using a connectivity-based unsupervised classifier. In: 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE), pp. 309–320. IEEE (2016)

  • Zhao, X., Wang, Z., Fan, X., Wang, Z.: A clustering-Bayesian network based approach for test case prioritization. In: 2015 IEEE 39th Annual Computer Software and Applications Conference, vol. 3, pp. 542–547. IEEE (2015)

  • Zimmermann, T., Premraj, R., Zeller, A.: Predicting defects for eclipse. In: Third International Workshop on Predictor Models in Software Engineering (PROMISE’07: ICSE Workshops 2007), p. 9. IEEE (2007)

Download references

Acknowledgements

The authors would like to thank Mrs. Monireh Ghasri for her helpful comments and tips and Dr. Hamid Beigi, Mr. Ahmad Khajehnejad, and members of the Intelligent Systems Laboratory (ISL) for kindly providing hardware facilities.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Computer Engineering Department, Sharif University of Technology, Azadi Ave, Tehran, Iran

    Mostafa Mahdieh, Seyed-Hassan Mirian-Hosseinabadi & Mohsen Mahdieh

Authors
  1. Mostafa Mahdieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seyed-Hassan Mirian-Hosseinabadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohsen Mahdieh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyed-Hassan Mirian-Hosseinabadi.

Ethics declarations

Conflict of interest

The author(s) declare(s) that there is no conflict of interest regarding the publication of this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahdieh, M., Mirian-Hosseinabadi, SH. & Mahdieh, M. Test case prioritization using test case diversification and fault-proneness estimations. Autom Softw Eng 29, 50 (2022). https://doi.org/10.1007/s10515-022-00344-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-022-00344-y

Keywords

Search

Navigation